In traditional photography, depth of field (DOF) is how much of your image is in focus. Shallow depth of field refers to when only things that are very close to the plane of the subject you focus on are in focus. Objects that are behind or in front of your subject will appear out of focus. Dynamic range refers to the difference between your highlights and shadows in an image. On average, the human eye can see dynamic range on the order of 1,000,000:1. What this means is that we can see details in both very bright and very dark areas of a scene at the same time. A digital sensor has a dynamic range on the order of 1,000:1, which means that if your subject is very bright compared to your background, when you expose the image correctly for your subject, you background will appear to be very dark. Conversely, if you exposed the image for the background, your subject may appear “blown out” or very bright white and overexposed.
As with traditional photography cameras, depth of field, i.e. image sharpness consistency within the camera Field of View (FOV) and range of interest, and image dynamic range, i.e. image contrast range within the camera FOV and range of interest, requirements can be a significant challenge to a depth camera. In the case of a depth camera, objects close to the camera's light source and or of high reflectivity can cause over saturation of sensor pixels while objects further from the camera's light source and or of low reflectivity can be difficult for the sensor/camera to detect at all.
A “Time-of-Flight” based depth camera comprises a depth image sensor, which is typically made using a standard CMOS fabrication process, and an IR light source for measuring distance, which is proportional to the length of time the IR light takes to travel from and return to the camera. A depth camera system generates depth images and transmits them to the host processor over a suitable, e.g. USB, interface.
The camera hardware includes a light source module, an IR light detecting, e.g. CMOS, image sensor, and an ambient light-color sensing, e.g. CMOS, image sensor. A 3D imager produces phase measurements that are processed either on sensor or in a remote coprocessor to produce actual range data. Such a camera can be used in “Z-only” mode for applications, which require the use of range data only. The camera could also be used in “RGB+Z”, i.e. full 3D depth and 2 dimensional colors, modes for applications which utilize both traditional color as well as depth images. Depth and color processing can be done in the camera or with a pass-through mode in which unprocessed data can be passed to the host for processing.
In a depth only camera, the sensor and light source will be synchronized in time. In RGB and depth cameras, the light source and the two sensors will be synchronized in time, such that both sensors start their frames cycles with a known and locked timing relationship, e.g. at the same time, with each other and the light source. Also, the frame start time of each sensor can be adjusted with respect to the data stream to the host to provide a system-level synchronization capability. Data from each sensor and audio can be transmitted to host devices on separate streams over various interfaces, such as a USB2.0 isochronous link, which may include a tagging capability to insert timestamps into each frame of each sensor. Similarly, the data streams could be integrated before transmission and de-integrated by the host.
The camera enables developers to create many new kinds of applications, e.g. gesture control of host devices, interactive games, etc., requiring both depth and color video.
In a typical Time of Flight (ToF) based camera, the camera is designed to synchronously modulate a light source at a fixed Peak Optical Power (POP) level with a sensor's active integration period, i.e. frame time. In some implementations, camera operating frequencies are varied in fairly narrow ranges, e.g. +/−20 MHz, to be able to detect distance aliasing artifacts caused by frequency wrap around or reflections of objects beyond the camera's working range created by frequency range multiples. In ToF based cameras, frequency is proportional to distance, e.g. an object at 6 m, which is 1 m outside a 5 m range of interest, can be falsely detected as an object of low reflectivity at 3 m, a distance within the range of interest.
An object of the present invention is to overcome the shortcomings of the prior art by providing a depth camera for a ToF system that divides the overall range of interest and Field of View (FoV) into Volume of Interest (VOI) sub-ranges with different frequency, peak optical power and integration period pairs or triplets for each VOI sub-range.